Preposition Disambiguation: Still a Problem

Considerable recent progress has been made in preposition disambiguation using the SemEval 2007 corpus, with results reaching accuracy of over 88 percent. However, with a new corpus of tagged instances, use of the models shows a decline in performance to around 43 percent. This suggests that recent efforts suffer from an out-of-domain problem. Detailed examination of the dimensions of this problem suggests that the sampling methodology used in creating the SemEval corpus was biased to the underlying reliance on FrameNet. In view of the demonstration of increasing importance attached to prepositions in semantic role labeling, further efforts to understand preposition behavior seem warranted. Initial examination of the new data may provide a basis for future directions in preposition disambiguation.